Method and device for detecting a shadow in a head mounted device

ABSTRACT

Disclosed is a method for detecting a shadow in an image of an eye region of a user wearing a Head Mounted Device, HMD. The method comprises obtaining, from a camera of the HMD, an image of the eye region of the user wearing a HMD and determining an area of interest in the image, the area of interest comprising a plurality of subareas. The method further comprises determining a first brightness level for a first subarea of the plurality of subareas and determining a second brightness level for a second subarea of the plurality of subareas. The method further comprises comparing the first brightness level with the second brightness level, and, based on the comparing, selectively generating a signal indicating a shadow.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims priority to Swedish Application No. 1851344-0,filed Oct. 29, 2018; the content of which are hereby incorporated byreference.

TECHNICAL FIELD

The present disclosure relates generally to methods, systems and devicesfor detecting a shadow in an image of a user wearing a head mounteddevice, and especially for detecting a shadow in the eye region of auser.

BACKGROUND

Today, eye-tracking technology is used for various differentapplications, such as e.g. Virtual Reality, VR, or Augmented Reality,AR. In some applications where eye-tracking is used, including VR and ARapplications, the user on whom the eye-tracking is performed is wearinga head mounted device, HMD. The HMD can in some cases have a display andcan in these cases also be referred to as a head mounted display.

In order to perform eye-tracking on a user wearing a HMD, movement ofthe user's eyes, including at least the pupils, needs to be detected andanalyzed. Tracking of the user's eye movements include detecting thepupil of a user, which may be done by using differences in contrast andbrightness levels between the pupil and other parts of the eye of auser, particularly the iris of a user. However, depending on factorssuch as e.g. where the cameras for detecting the eye movement aremounted and where illuminators providing light are mounted, shadows canoccur which make it more difficult to accurately detect the eyemovements of a user due to the decrease in difference of contrast and/orbrightness levels between the pupil and the surrounding parts of the eyeregion. This is especially a problem in the case where a shadow wouldappear that is in the same area as the pupil of a user.

Consequently, there exists a need for improvement when it comes toaccurately tracking the eye movements of a user wearing a HMD,especially wherein the configuration of the HMD and/or the positioningof the HMD on a user is such that shadows appear in the eye region of auser.

SUMMARY

It is an object of the invention to address at least some of theproblems and issues outlined above. An object of embodiments of theinvention is to provide a device and a method which improves the qualityand accuracy of eye-tracking for a user wearing a Head Mounted Device,HMD. It may be possible to achieve these objects, and others, by usingmethods, devices and computer programs as defined in the attachedindependent claims.

According to one aspect a method is provided for detecting a shadow inan image of an eye region of a user wearing a HMD. The method comprisesobtaining, from a camera of the HMD, an image of the eye region of theuser wearing a HMD and determining an area of interest in the image, thearea of interest comprising a plurality of subareas. The method furthercomprises determining a first brightness level for a first subarea ofthe plurality of subareas and determining a second brightness level fora second subarea of the plurality of subareas. The method furthercomprises comparing the first brightness level with the secondbrightness level, and based on the comparing, selectively generating asignal indicating a shadow.

According to another aspect, a HMD is provided. The HMD comprisesprocessing circuitry and a memory. The memory contains instructionsexecutable by the processing circuitry, whereby the HMD is operative forobtaining, from a camera of the HMD, an image of the eye region of theuser wearing a HMD. The HMD is further operative for determining an areaof interest in the image, the area of interest comprising a plurality ofsubareas. HMD is further operative for determining a first brightnesslevel for a first subarea of the plurality of subareas and determining asecond brightness level for a second subarea of the plurality ofsubareas. The HMD is further operative for comparing the firstbrightness level with the second brightness level, and based on thecomparing, selectively generating a signal indicating a shadow.

According to other aspects, computer programs and carriers are alsoprovided, the details of which will be described in the claims and thedetailed description.

Further possible features and benefits of this solution will becomeapparent from the detailed description below.

BRIEF DESCRIPTION OF DRAWINGS

The solution will now be described in more detail by means of exemplaryembodiments and with reference to the accompanying drawings, in which:

FIG. 1 shows a head mounted device according to an embodiment of thepresent disclosure.

FIG. 2 shows an image of the eye region of a user of a HMD.

FIG. 3 is a flow chart schematically showing method steps of anembodiment of a method according to the present disclosure.

FIG. 4 shows a block schematic of a head mounted display according to anembodiment.

DETAILED DESCRIPTION

Briefly described, a method for detecting a shadow in an eye region of auser wearing a Head Mounted Device, HMD, is provided, which improves theaccuracy and the quality of eye-tracking that can be performed for theuser of the HMD. In this patent application, shadow is defined asinsufficient illumination for eye feature detection, such as detectionof the pupil. The method comprises obtaining an image of the eye regionof the user wearing the HMD and determining an area of interest in theimage, wherein the area of interest comprises a plurality of subareas.The area of interest is in one embodiment the middle part of the eyeregion of a user, including at least the area surrounding and comprisingthe pupil. After obtaining the image, the method comprises determining afirst brightness level for a first subarea, which in one embodiment is asubarea comprising at least the pupil of the user, which may alsocontain a shadow. The method further comprises determining a secondbrightness level for a second subarea, wherein the second subarea is inone embodiment an area not comprising the pupil of a user. The methodfurther comprises comparing the first brightness level with the secondbrightness level and based on the comparing, selectively generating asignal indicating a shadow. In one embodiment the signal indicating ashadow is generated if the comparing indicates that there is a shadow,and no signal is generated of the comparing indicates that there is nota shadow. In one embodiment, the comparing indicates that there is ashadow if the difference in brightness levels for the first and secondsubareas is above a certain threshold, wherein the threshold may bepredetermined or it may be adapted for the user wearing the HMD.

Methods for eye-tracking commonly use differences in contrast andbrightness level in an image, typically at least between the iris andthe pupil of a user, in order to determine where the pupil of a user is.An insight relevant to the present disclosure is that accuratelydetecting the location of the pupil of a user wearing a HMD can becomedifficult in case the HMD causes shadows to appear in the eye region ofa user, especially when such shadows are located in the same area as thepupil of the user. A reason for shadows interfering with theeye-tracking capabilities of a HMD, especially when they are in thepupil area, is that they decrease the difference in brightness betweenthe pupil area and the surrounding area, and this difference is oftenused in order to track the pupil. Shadows in a HMD often appear asvertical shadows, which may cover e.g. the middle of the eye region ofthe user, which is typically the area in which the pupil is mostcommonly located. Reasons for unwanted shadows appearing in the eyeregion include that HMDs often have illuminators placed at an angle infront of the eyes of the user. Preferably the illuminators arepositioned on a frame surrounding a lens or a hole that during use ofthe HMD is positioned in front of the eyes or each of the eyes. Thelarger the lens or hole is, the higher the incidence angle would be ofthe light from the illuminators towards the eye(s). The higher theincidence angle is the higher the risk is that the eye will be shadowed.In cases where a HMD does produce shadows in the eye region of a user,this may result in worse conditions for eye-tracking. However, theproblem can usually be solved by adjusting the position of the HMD. Theposition of the HMD can in some embodiments be adjusted by changingsettings of the HMD, such as the settings of an eye-relief of the HMD,i.e. the distance between the eyes and the HMD. This distance may be adistance between each of the eyes and a respective lens of the HMD. Insome embodiments, the HMD can be adjusted automatically by the HMD, andin some embodiments the user would need to manually adjust it after ithas been indicated to the user that there is a shadow. By using methods,devices and computer programs according to the present disclosure, theaccuracy and quality of eye-tracking can be improved by detecting ashadow covering the pupil of a user that would decrease the quality andaccuracy of the eye-tracking, and selectively generating a signal toindicate a shadow in case one is detected.

FIG. 1 shows an example of a Head Mounted Device, HMD 1, which may beused in methods, devices and computer programs according to the presentdisclosure. As will be understood, the HMD 1 of FIG. 1 is merely anexample, any HMD comprising or being capable of performing at least thefeatures of the attached independent claims does fulfill the criteria ofan HMD of the present disclosure. The HMD 1 comprises at least onecamera 10, adapted to capture an image of the eye region of a userwearing the HMD. The camera 10 may be placed anywhere on or operativelyconnected to the HMD, as long as it is able to obtain an image of theeye region of a user wearing the HMD. Common placements of cameras 10are in positions that would be in front of and slightly below the eye ofa user when the user is wearing the HMD 1. In some embodiments, the HMDmay comprise multiple cameras 10, for example there may be one camera 10for each eye of the user. In some embodiments, the camera 10 may bemounted on a HMD 1 which was not manufactured with a camera 10 to startwith. The HMD may also comprise illuminators (not shown) which areadapted to provide illumination of the eye region of a user wearing theHMD. The placement of the illuminators of a HMD may be relevant towhether a shadow is generated by the HMD 1 when a user is wearing theHMD, as well as HMD design and the characteristics and facial shape ofthe user wearing the HMD 1. The HMD 1 may optionally comprise at leastone display 30. Further, the HMD 1 may comprise at least one lens 40.Further, in some embodiments the HMD 1 may optionally comprise at leastone hot mirror 50. However, the hot mirror may be replaced by aholographic film or something completely different.

Looking now at FIG. 2 , an image 100 of an eye region of a user isshown. The image is typically obtained from a camera of the HMD 1. Thecamera of the HMD 1 may be a camera contained in, mounted on oroperatively connected to the HMD, and may also be obtained by a cameramounted on the HMD which was not a part of the HMD as manufactured. Theimage 100 comprises a plurality of pixels, which may be seen as aplurality of vertical columns of pixels, extending along the Y-axis ofthe image 100, and a plurality of horizontal rows of pixels, extendingalong the X-axis of the image 100. The image 100 comprises an area ofinterest 130, wherein the area of interest is in in one embodiment anarea surrounding and comprising the pupil 120 of the user. In someembodiments, the area of interest may be predetermined as a certain partof the image 100, for example the middle 50% of the image. In someembodiments, the area of interest may be dynamically determined, forexample by using image recognition technology on images obtained of theeye region of a user in order to determine which area comprises the areasurrounding and comprising the pupil 120 of a user. If a shadow ispresent in an image, such as the shadow covering the middle part of theimage 100 of FIG. 2 , it may become difficult to accurately detect thelocation of the pupil 120 of the user. This can result in, for example,that a pupil is erroneously determined to be located at a position 110which is not the actual position of the pupil 120.

When looking to detect a shadow in an image 100 of an eye region of auser, the difference in brightness level between different subareas ofthe image may be used. A first subarea 140 is in one embodiment chosenas a part of the image comprising at least the pupil of the user, and asecond subarea 150, 160 is chosen as a part of the image not comprisingthe pupil. Due to the pupil being black, the subarea 140 comprising thepupil is expected to have a lower brightness level than the subarea 150,160 not comprising the pupil. Further, if there is a shadow present inthe subarea 140 comprising the pupil, and no shadow or less shadow ispresent in the subarea 150, 160 not comprising the pupil, the differencein brightness level between the first subarea 140 and the second subarea150, 160 is larger than in cases where there is no shadow in the pupilarea of a user. By looking at the differences in brightness values in animage of an eye region of a user of a HMD, such as the differencebetween the highest brightness value and the lowest brightness value ofthe image, and comparing these differences with differences in images ofeye regions with no shadows present, it can be determined whether thereis a shadow in an image, or if there is likely to be a shadow in animage. This insight is used in the present disclosure to determinewhether there is a shadow in the first subarea 140.

FIG. 3 , in conjunction with FIGS. 1 and 2 , shows an embodiment of amethod for detecting a shadow in the eye region of a user wearing a HMD1, wherein the HMD 1 comprises a camera 10. The method may in someembodiments be performed by the HMD 1, and in some embodiments themethod may be performed elsewhere using data obtained from the HMD 1,for example in a cloud environment to which the HMD 1 is operativelyconnected. The method comprises obtaining 202 an image of an eye regionof a user wearing the HMD 1. The method further comprises determining204 an area of interest 130 in the obtained image, wherein the area ofinterest 130 comprises a plurality of subareas. The method furthercomprises determining 206 a first brightness level for a first subarea140 of the image. The first subarea 140 may for example be one or morecolumns of pixels of the image, or one or more rows of pixels of theimage. In some embodiments, the first subarea 140 is the area comprisingthe lowest brightness value in the image, which is typically the areacomprising 140 the pupil of the user, wherein the brightness value islower if the first subarea 140 comprises a shadow than if the firstsubarea 140 does not comprise a shadow. The method further comprisesdetermining 208 a second brightness level for a second subarea 150, 160of the image. The second subarea 150, 160 may for example be one or morecolumns of pixels of the image, or one or more rows of pixels of theimage. In some embodiments, the second subarea 150, 160 is an areahaving the highest brightness value in the image, which is typically anarea not comprising the pupil of a user. In some embodiments, the firstsubarea 140 is adjacent to the second subarea 150, 160. The methodfurther comprises comparing 210 the first brightness level with thesecond brightness level, and based on the comparing, selectivelygenerating 212 a signal indicating a shadow. In some embodiments,selectively generating 212 a signal comprises generating a signal incase the comparing is above a certain threshold and not generating asignal in case the comparing is below a certain threshold.

In some embodiments, the step of obtaining 202 an image of the eyeregion of a user comprises that a camera 10 of the HMD 1 captures animage of the eye region of a user wearing the HMD. In some embodiments,obtaining 202 an image of the eye region of a user comprises obtainingmultiple images from different cameras, and combining the multipleimages into one image. In some embodiments, obtaining 202 an image ofthe eye region of a user comprises obtaining multiple images from thesame camera, and combining the multiple images into one image. Theobtained image 100 is in a preferred embodiment a picture in black andwhite, and the step of obtaining 202 an image of the eye region of auser may in some embodiments include converting a captured image intoblack and white.

In some embodiments, the area of interest may be the entire image 1. Insome embodiments, the area of interest may be the area surrounding andcomprising the pupil of a user. In some embodiments, the size and/orposition of the area of interest may be predetermined, and in someembodiments the area of interest may be dynamically determined, forexample the area of interest may be determined in a user-specific way.In some embodiments, the step of determining 204 an area of interest 130in the image 100 comprises choosing a number of columns of pixelscorresponding to the middle 50% of the image. In some embodiments,determining 204 an area of interest 130 in the image 100 compriseschoosing a number of columns of pixels corresponding to the area inwhich the pupil of the user is located. In some embodiments, determining204 an area of interest 130 in the image 100 comprises choosing a numberof columns of pixels corresponding to the area in which the entire eyeof the user is located. The area of interest may be virtually any partof the obtained image, but it is typically an area comprising at leastthe area surrounding and comprising the pupil of the user. In someembodiments, determining an area of interest is done in order to excludeparts of the image which has constant dark pixels, denoted as 170 inFIG. 2 , which may be caused by e.g. lens cups of the HMD. In someembodiments, determining an area of interest comprises excluding allparts of the image containing a dark group of pixels 170 which are inthe periphery of the image.

In some embodiments, determining 206 a first brightness level for afirst subarea 140 of the image and/or determining 208 a secondbrightness level for a second subarea 150, 160 of the image, comprisesmeasuring and/or determining the brightness levels of the pixels in thefirst and second subarea 150, 160 s, respectively. In one embodiment,the first subarea 140 is chosen as an area comprising the darkest partof the image, i.e. a part with the lowest brightness value. The secondsubarea 150, 160 is in one embodiment chosen as an area comprising thebrightest part of the image, i.e. a part with the highest brightnessvalue. In some embodiments, the first and second subareas are chosensuch that the first subarea 140 has a lower brightness value than thesecond subarea 150, 160. The brightness value may in some embodiments bemeasured by calculating vertical projections of the image, wherein avertical projection is the sum of pixel intensity values in a subarea,calculated as a column from the top the bottom. The pixel intensityvalues may in some embodiments range from 0 to 255, wherein 0 denotesblack and 255 denotes white in a black and white image. In someembodiments, measuring the brightness value may further comprisecalculating a pixel intensity value for the first subarea 140, which maybe expressed as e.g. an average pixel intensity value or as a totalpixel intensity value. As will be understood, this can be done invarious ways, e.g. by summing up the pixel intensity value for eachindividual pixel in the first subarea 140 and then optionally dividingit by the number of pixels in the first subarea 140. In someembodiments, the vertical projections may be calculated as follows,wherein the image has a width=m and a height=n, and wherein the verticalprojection is a vector of size m:vertical_projection[i]=Σ_(j=0) ^(n-1)pixel(i,j)

In some embodiments, determining the brightness level of a subarea isbased on brightness levels measured and/or recorded over a period oftime. By basing the brightness value on values measured over time, itbecomes easier to eliminate noise in the image, such as e.g.reflections. If a series of images are captured of the same eye regionof a user wearing the HMD over time, the vertical projection of onearea, such as the x:th column in one image, should correspond to thesame area of the user's eye as the x:th column in another image of thesame eye region. In some embodiments, the brightness level of an area isdetermined as the average brightness value of the series of images takenover time. In some embodiments, the brightness level of a subarea isdetermined as the maximum brightness value of the series of images takenover time. In some embodiments, the brightness level of an area isdetermined as the minimum brightness value of the series of images takenover time. By using the maximum and/or minimum brightness level of aseries of images recorded over time, it becomes possible to eliminatenoise caused by e.g. the eye moving around. Since shadow detection istypically based on differences in brightness levels, and the pupil of auser is dark, the difference in brightness levels between two subareaschange if the pupil of the user moves. However, the shadow wouldgenerally be constant, which entails that the brightness level for asubarea comprising a shadow is always low, but outside of the shadow thebrightness level would only be low in some eye positions, which wouldnot be constant, at least not in cases where the shadow is located inthe area comprising the pupil of a user.

Always keeping a maximum value for a subarea, for example keeping amaximum value for each vertical projection, could result in that themethod still indicates that there is a shadow even after the HMD isadjusted such that the shadow disappears. This may be solved by, in someembodiments, keeping the maximum value for each subarea, but also havingthose maximum values decay over time, and always using the highestrecorded value of the most recently recorded value and the most recentmaximum with a decay applied. In some embodiments, the maximum valuedecays with 5% for each frame, in an implementations where 90 frames persecond are obtained.

In some embodiments where a series of images is recorded over time, themethod may also comprise using a filter for determining whether a shadowis present or not. A filter may be implemented such that a certainpercentage of the most recently recorded images need to indicate thatthere is a shadow, in order for the method to generate a signalindicating that there is a shadow. In some embodiments, the method needsto indicate that there is a shadow present for 75/100 of the mostrecently recorded images in order for the method to indicate that thereis a shadow. By applying such a filter, the effect of quick variationsin shadow and brightness caused by e.g. moving the eyes around, movingthe HMD around, adjusting the illuminators, etc., can be decreasedand/or eliminated.

In some embodiments, the method further comprises, prior to determining204 an area of interest 130 in the image 100, defining a region ofinterest of the image, and also determining if the area of interest 130is within the region of interest. In some embodiments, the subsequentsteps are then only performed in the area of interest 130 is determinedto be within the region of interest. The region of interest may in someembodiments be the entire image. The region of interest may in someembodiments be the area in which the pupil of the user is expected tobe, which may be predetermined or user-specific. In some embodiments,where the pupil is expected to be is based on previously acquired imagesfrom the same user, and in some embodiments it is based on imagesacquired from other users than the user currently using the HMD.

In some embodiments, the method may further comprise a step ofdetermining if a shadow is likely to appear as a horizontal shadow or avertical shadow. Such information may be used e.g. such that verticalprojections are used for determining brightness values in case avertical shadow is expected, and horizontal projections are used fordetermining brightness values in case a horizontal shadow is expected.Calculating horizontal projections would then be done in the same was asfor the vertical projections as described above, but instead of summingthe pixels from top to bottom for each column, the pixels are summedfrom left to right for each row. Determining if a shadow is likely toappear as a horizontal shadow or a vertical shadow may compriseanalyzing a number of images taken over time and detecting if and wherea shadow is located in those images, and then determining that a shadowis most likely to appear in the same direction as in those images.

In some embodiments, comparing the first brightness level with thesecond brightness level comprises comparing the brightness level of thepixels of the first subarea 140 with the brightness level of the pixelsof the second subarea 150, 160. In some embodiments, the brightnesslevels of the pixels in a subarea correspond to the vertical projectionsof that subarea, as defined above. In some embodiments, the comparingcomprises subtracting the total or average pixel intensity value of thefirst subarea 140 from the total or average pixel intensity value of thesecond subarea 150, 160. In some embodiments, the comparing 210comprises applying a function over the first and second brightnesslevels. Such a function may be any kind of function, such as atransform.

In some embodiments, the step of selectively generating 212 a signalindicating a shadow comprises generating a signal if the comparingindicates that the difference between the first brightness value and thesecond brightness value is large enough. In practice, this would usuallyentail that the first subarea 140 is substantially darker than thesecond subarea 150, 160, which in turn would entail that it is likelythat there is a shadow present in the first subarea 140. Determiningthat the difference between the first brightness level and the secondbrightness level is large enough may comprise comparing the differencewith a predetermined threshold, and if the difference is above thethreshold it is determined that the difference is large enough togenerate a signal indicating a shadow, and if the difference is belowthe threshold no signal is generated. In some embodiments, thepredetermined threshold is based on differences in brightness levelsbetween first and second subareas of images where no shadow is present.

In some embodiments, the threshold level may be user-specific. In suchcases, determining a user-specific threshold level may entaildetermining a brightness value for the user's eye region, which may insome embodiments be an average brightness value. In some embodiments,determining a user-specific threshold level may comprise determiningbrightness level(s) of one or more subareas of the image. In someembodiments, a function is applied over the brightness levels whendetermining a user-specific threshold. In some embodiments, the averagebrightness value for the user's eye region used for this threshold doesnot include the pupil and/or does not include the entire eye of theuser. In some embodiments, the threshold may be based on a measure ofthe eye feature detection quality or eye feature detection performance.By taking the eye feature detection quality and/or performance intoaccount, the threshold levels may for example be adapted such thatdevices with a higher performance/quality has a higher requiredthreshold value than devices with a lower quality of the eye detectionfeature.

In some embodiments, the signal which may be generated in the step ofselectively generating 212 a signal indicating a shadow, furthercomprises an instruction for adjustment of the HMD. The instruction mayin some embodiments be an instruction to adjust the position of the HMD,which may include instructions for adjusting an eye-relief of the HMD.In some embodiments, the instruction may further comprise informationregarding how the position of the HMD should be adjusted. Depending onthe capabilities of the HMD, the instruction may be automaticallyexecuted by the HMD if such functionality exists, and if no HMDfunctionality exists for automatically adjusting the HMD, then theinstruction may be intended as information to the user of the HMD, suchthat the user can determine what to do based on the instruction. In someembodiments, the instruction may comprise information regarding how toreposition and/or readjust illuminators of the HMD.

In some embodiments, the method may further comprise determining a thirdbrightness level for a third subarea, and the comparing 210 may furthercomprise comparing the third brightness level with the first and secondbrightness levels.

FIG. 4 , in conjunction with FIG. 1 , shows a HMD 1, capable ofperforming eye-tracking on a user wearing the HMD 1. The HMD 1 comprisesprocessing circuitry 603 and a memory 604. The processing circuitry 603may comprise one or more programmable processor, application-specificintegrated circuits, field programmable gate arrays or combinations ofthese (not shown) adapted to execute instructions. The memory containsinstructions executable by said processing circuitry, whereby the HMD 1is operative for obtaining, from a camera of the HMD, an image of theeye region of the user wearing the HMD 1. The HMD 1 is further operativefor determining an area of interest in the image, wherein the area ofinterest comprises a plurality of subareas. The HMD 1 is furtheroperative for determining a first brightness level for a first subareaof the plurality of subareas, and determining a second brightness levelfor a second subarea of the plurality of subareas. The HMD 1 is furtheroperative for comparing the first brightness level with the secondbrightness level, and based on the comparing, selectively generating asignal indicating a shadow.

According to an embodiment, the first subarea is a subarea having ahigher brightness level than other subareas, and may be the subarea withthe highest brightness level. In some embodiments, the second subarea isa subarea having a lower brightness level than other subareas, and maybe the subarea with the lowest brightness level.

According to an embodiment, the comparing comprises subtracting thefirst brightness level from the second brightness level.

According to an embodiment, the signal which is selectively generatedcomprises an instruction for adjustment of the HMD. The instruction maybe an instruction for the HMD 1 or for a user of the HMD 1.

According to an embodiment, the steps of determining brightness levelsfor the first and second subareas is based on brightness levels recordedover a predetermined period of time.

According to an embodiment, the signal is generated if the comparing isabove a threshold value. In some embodiments, no signal is generated ifthe comparing is below a threshold value.

According to an embodiment, the HMD 1 is further operative for applyinga function over the first and second brightness levels, or one of them,before the comparing step.

According to an embodiment, determining a first brightness level for afirst subarea of the plurality of subareas and determining a secondbrightness level for a second subarea of the plurality of subareas,comprises comparing the brightness levels of the pixels of the first andsecond subareas, respectively

According to an embodiment, the image comprises a plurality of columnsof pixels, and each subarea corresponds to one or more columns ofpixels.

According to an embodiment the HMD 1 is further operative for, prior todetermining an area of interest in the image defining a region ofinterest of the image, and for, after the determining an area ofinterest in the image, determining if the area of interest is within theregion of interest. According to an embodiment, the subsequent steps arethen only performed if the area of interest is within the region ofinterest.

According to an embodiment, the HMD 1 is further operative fordetermining a third brightness level for a third subarea, wherein thecomparing further comprises comparing the third brightness level withthe first and second brightness levels.

In some embodiments, the feature(s) of the HMD, e.g. the processingcircuitry and the memory, which perform the method steps may be a groupof network nodes, wherein functionality for performing the method arespread out over different physical, or virtual, nodes of the network. Inother words, the feature(s) of the HMD which perform the method stepsmay be a cloud-solution, i.e. the feature(s) of the HMD which performthe method steps may be deployed as cloud computing resources that maybe distributed in the network.

According to other embodiments, the HMD 1 may further comprise acommunication unit 602, which may be considered to comprise conventionalmeans for communicating with relevant entities, such as other computersor devices to which it is operatively connected. The instructionsexecutable by said processing circuitry 603 may be arranged as acomputer program 605 stored e.g. in the memory 604. The processingcircuitry 603 and the memory 604 may be arranged in a sub-arrangement601. The sub-arrangement 601 may be a micro-processor and adequatesoftware and storage therefore, a Programmable Logic Device, PLD, orother electronic component(s)/processing circuit(s) configured toperform the methods mentioned above.

The computer program 605 may comprise computer readable code means,which when run in a HMD 1 causes the HMD 1 to perform the stepsdescribed in any of the described embodiments of the HMD 1. The computerprogram 605 may be carried by a computer program product connectable tothe processing circuitry 603. The computer program product may be thememory 604. The memory 604 may be realized as for example a RAM(Random-access memory), ROM (Read-Only Memory) or an EEPROM (ElectricalErasable Programmable ROM). Further, the computer program may be carriedby a separate computer-readable medium, such as a CD, DVD or flashmemory, from which the program could be downloaded into the memory 604.Alternatively, the computer program may be stored on a server or anyother entity connected to the HMD, to which the HMD has access via thecommunication unit 602. The computer program may then be downloaded fromthe server into the memory 604.

Although the description above contains a plurality of specificities,these should not be construed as limiting the scope of the conceptdescribed herein but as merely providing illustrations of someexemplifying embodiments of the described concept. It will beappreciated that the scope of the presently described concept fullyencompasses other embodiments which may become obvious to those skilledin the art, and that the scope of the presently described concept isaccordingly not to be limited. Reference to an element in the singularis not intended to mean “one and only one” unless explicitly so stated,but rather “one or more”. All structural and functional equivalents tothe elements of the above-described embodiments that are known to thoseof ordinary skill in the art are expressly incorporated herein byreference and are intended to be encompassed hereby. Moreover, it is notnecessary for an apparatus or method to address each and every problemsought to be solved by the presently described concept, for it to beencompassed hereby. In the exemplary figures, a broken line generallysignifies that the feature within the broken line is optional.

The invention claimed is:
 1. A method for detecting a shadow in an imageof an eye region of a user wearing a Head Mounted Device (HMD)comprising: obtaining, from a camera of the HMD, an image of the eyeregion of the user wearing the HMD; determining an area of interest inthe image, wherein the area of interest comprises a plurality of columnsof pixels corresponding to an area in the image in which an entire eyeof the user is located; determining a first subarea within the area ofinterest, the first subarea comprising a first set of the columns ofpixels having a first brightness level, wherein the first brightnesslevel is calculated as a sum of pixel intensity values in the first setof the columns of pixels; determining a second subarea within the areaof interest, the second subarea comprising a second set of the columnsof pixels having a second brightness level, wherein the secondbrightness level is calculated as a sum of pixel intensity values in thesecond set of the columns of pixels; comparing the first brightnesslevel with the second brightness level; and when the difference betweenthe first brightness level and second brightness level is greater than apredetermined threshold, generating a signal indicating a presence of ashadow.
 2. The method according to claim 1, wherein the first brightnesslevel is the lowest brightness level in the area of interest, and thesecond brightness level is the highest brightness level in the area ofinterest.
 3. The method according to claim 1, wherein the comparingcomprises subtracting the first brightness level from the secondbrightness level.
 4. The method according to claim 1, wherein saidsignal further comprises an instruction for adjustment of the HMD. 5.The method according to claim 1, wherein determining the firstbrightness level and the second brightness level is based on brightnesslevels recorded over a predetermined period of time.
 6. The methodaccording to claim 1, further comprising applying a function over thefirst brightness level and the second brightness level before thecomparing.
 7. The method according to claim 1, further comprising: priorto determining the area of interest in the image, defining a region ofinterest of the image; and after the determining the area of interest inthe image, determining that the area of interest is within the region ofinterest.
 8. The method according to claim 7, wherein the region ofinterest comprises an area in which a pupil of the user is expected tobe based on analysis of previously acquired images of the eye region ofthe user.
 9. The method according to claim 7, wherein the region ofinterest comprises an area in which a pupil of the user is expected tobe based on analysis of previously acquired images of eye regions ofother users.
 10. The method according to claim 1, further comprisingdetermining a third subarea within the area of interest, the thirdsubarea comprising a third set of the columns of pixels having a thirdbrightness level, wherein the third brightness level is calculated as asum of pixel intensity values in the third set of the columns of pixels;and wherein the comparing further comprises comparing the thirdbrightness level with the first brightness level and the secondbrightness level.
 11. A head mounted device (HMD) adapted to be worn bya user, the HMD comprising: at least one camera; processing circuitry;and a memory, said memory containing instructions executable by saidprocessing circuitry, whereby said HMD is operative for: obtaining, froma camera of the HMD, an image of the eye region of the user wearing theHMD; determining an area of interest in the image, wherein the area ofinterest comprises a plurality of columns of pixels corresponding to anarea in the image surrounding and including a pupil of the user;determining a first subarea within the area of interest, the firstsubarea comprising a first set of the columns of pixels having a firstbrightness level, wherein the first brightness level is calculated as asum of pixel intensity values in the first set of the columns of pixels;determining a second subarea within the area of interest, the secondsubarea comprising a second set of the columns of pixels having a secondbrightness level, wherein the second brightness level is calculated as asum of pixel intensity values in the second set of the columns ofpixels; comparing the first brightness level with the second brightnesslevel; and when the difference between the first brightness level andsecond brightness level is greater than a predetermined threshold,generating a signal indicating a presence of a shadow.
 12. The HMDaccording to claim 11, further comprising applying a function over thefirst brightness level and the second brightness level before thecomparing.
 13. The HMD according to claim 11, wherein the firstbrightness level is the lowest brightness level in the area of interest,and the second brightness level is the highest brightness level in thearea of interest.
 14. The HMD according to claim 11, further comprising:prior to determining an area of interest in the image, defining a regionof interest of the image; and after the determining an area of interestin the image, determining that the area of interest is within the regionof interest.
 15. The HMD according to claim 14, wherein the region ofinterest comprises an area in which a pupil of the user is expected tobe based on analysis of previously acquired images of the eye region ofthe user.
 16. The HMD according to claim 14, wherein the region ofinterest comprises an area in which a pupil of the user is expected tobe based on analysis of previously acquired images of eye regions ofother users.
 17. The HMD according to claim 11, further comprisingdetermining a third subarea within the area of interest, the thirdsubarea comprising a third set of the columns of pixels having a thirdbrightness level, wherein the third brightness level is calculated as asum of pixel intensity values in the third set of the columns of pixels;and wherein the comparing further comprises comparing the thirdbrightness level with the first brightness level and the secondbrightness level.
 18. A non-transitory computer readable medium storinginstructions that, upon execution by a Head Mounted Device (HMD) causethe HMD to perform the following steps: obtaining, from a camera of theHMD, an image of the eye region of the user wearing a HMD; determiningan area of interest in the image, wherein the area of interest comprisesa plurality of columns of pixels corresponding to an area in the imagesurrounding and including a pupil of the user; determining a firstsubarea within the area of interest, the first subarea comprising afirst set of the columns of pixels having a first brightness level,wherein the first brightness level is calculated as a sum of pixelintensity values in the first set of the columns of pixels; determininga second subarea within the area of interest, the second subareacomprising a second set of the columns of pixels having a secondbrightness level, wherein the second brightness level is calculated as asum of pixel intensity values in the second set of the columns ofpixels; comparing the first brightness level with the second brightnesslevel; and when the difference between the first brightness level andsecond brightness level is greater than a predetermined threshold,generating a signal indicating a presence of a shadow.
 19. Thenon-transitory computer readable medium according to claim 18, whereinthe first brightness level is the lowest brightness level in the area ofinterest, and the second brightness level is the highest brightnesslevel in the area of interest.
 20. The non-transitory computer readablemedium according to claim 18, wherein the instructions, upon executionby the HMD, further cause the HMD to perform the step of determining athird subarea within the area of interest, the third subarea comprisinga third set of the columns of pixels having a third brightness level,wherein the third brightness level is calculated as a sum of pixelintensity values in the third set of the columns of pixels; and whereinthe comparing further comprises comparing the third brightness levelwith the first brightness level and the second brightness level.